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UNIFIED IMAGE PROCESSING SYSTEM 

CROSS REFERENCE TO RELATED APPLICATIONS 
[0001] This application claims priority to U.S. Provisional Patent Application No. 60/448,080, 
filed February 20, 2003 for Unified Image Processing System, the entire disclosure of which is 
incorporated by reference for all purposes. 

ATTACHED DOCUMENTS PART OF THIS APPLICATION 
[0002] The following documents are attached to this application and are considered part of this 
application: 

1 . Copy of U.S. Provisional Patent Application No. 60/448,080, filed February 20, 2003 for 
Unified Image Processing System. 16 pages. 

2. HTML user interface example. 1 page. 

3. HTML user interface example source code. 10 pages. 

4. Text Form user interface example. 1 page. 

5. Parameter Array File Diagnostic Form. 1 page. 

6. Text Form user interface to Parameter Array File source code. 10 pages. 

7. Parameter Array composition source code. 7 pages. 

8. Parameter Array to Diagnostic Form source code. 9 pages. 

9. Parametric Image Transform Execution source code. 17 pages. 

10. "Parametric Transforms." 4 pages. 

BACKGROUND OF THE INVENTION 
[0003] During image processing and manipulations, problems with processed images, which are 
referred to as pathologies, arise. In most cases, it is up to the user to avoid pathologies by visual 
feedback and interactively moving a control to adjust the amount of processing tolerated by a 
particular image. The control settings are generally a compromise between increasing the 
desirable results and keeping the pathologies acceptably low. 



[0004] Some examples of image pathologies are as follows. A typical "sharpen" filter causes 
halo and outlining effect, and may further enhance jpeg artifacts, particularly in relatively 
uniform areas such as sky. Further, burn out and color shift are often caused by brightness 
adjustment. Posterization and localized color shifts are caused by excessive processing. 
5 Textured objects are "flattened" by low local contrast in part of the transform. 

SUMMARY OF THE INVENTION 
[0005] According to one aspect, the invention provides an image processing method that 
comprises generating a synthesized image transform from a set of synthesizing parameters; and 
subjecting the input image to the synthesized image transform to form an output image wherein: 

10 the synthesized image transform is generated from a set of synthesizing parameters; the 
synthesizing parameters have a defined domain of allowed values; the complete set of 
synthesized transforms is obtained by varying the synthesizing parameters over all values within 
the domain; the visual effect of each synthesized transform is characterized by a set of visual 
effect parameters; and the visual effect parameters and the synthesizing parameters are related in 

15 that the visual effect parameters are generated from the synthesizing parameters for the complete 
set of transforms, and/or the synthesizing parameters are generated from the visual effect 
parameters for the complete set of transforms. 

[0006] In some embodiments, the synthesizing parameter sets are composable (that is, a function 
or algorithm combines two or more sets of synthesizing parameters into a single set of 
20 synthesizing parameters); and the composed synthesizing parameters are sequentially equivalent 
(that is, an image processed sequentially by two or more synthetic transforms will be equivalent 
to an image processed through a single transform which has been synthesized from the 
synthesizing parameter set composed from the synthesizing parameter sets of the sequential 
transforms). 

25 [0007] Embodiments of the invention are capable of eliminating or significantly reducing 
pathologies, while simplifying the optimization of images, thereby removing the need for 
experienced operators and interactive controls. The transforms also produce superior images 
without degrading the data from the original image. This feature allows an image to undergo 
further processing, while eliminating the occurrence of further pathologies. 
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[0008] A further understanding of the nature and advantages of the present invention may be 
realized by reference to the remaining portions of the specification and the drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0009] FIG. 1 shows an image transform; 

5 [0010] FIG. 2 shows a parametric image transform; 

[0011] FIG. 3 shows image visual effect functions and parameters; 

[0012] FIG. 4 shows transform visual effect parameters; 

[0013] FIG. 5 shows transform parameter set and visual effect parameter set conversion; 

[0014] FIG. 6 shows composing a sequence of parametric transforms into a single transform; 
10 [0015] FIG. 7 shows scalar composition; 

[0016] FIG. 8 shows simulation of a sequence of arbitrary transforms; 

[0017] FIG. 9 shows the simulation of a sequence of arbitrary transforms; 

[0018] FIG. 10 shows the synthesis of a sequence of visual intents; 

[0019] FIG. 1 1 shows the execution of a selective image transform; 
15 [0020] FIG. 12 shows the derivation of a v-array; and 

[0021] FIG. 13 shows a program/file overview. 

DESCRIPTION OF SPECIFIC EMBODIMENTS 
[0022] FIG. 1 shows an image transform. Page 5 of the Provisional Application, section 
"Fundamental Definition and Algorithms," subsection "Image transform" presents a notation for 
20 an arbitrary image transform. Note that the notation x' = t(x) is equivalent to: 

r'=t r (r,g,b) 

g'=t g (r,g,b) 

b'=t b (r,g,b) 

[0023] FIG. 2 shows a parametric image transform. Page 8 of the Provisional Application, 
25 section "Fundamental Definition and Algorithms," subsection "Parametric transforms" presents 
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the notation and definition for a parametric transform. We also refer to a parametric transform as 
a "synthesized" transform. 

[0024] FIG. 3 shows image visual effect functions and parameters. Page 7 of the Provisional 
Application, section "Fundamental Definition and Algorithms," subsection "Analytic selective 
5 transforms" presents the notation land definition for a set of analytic selective functions. The 
selective functions are also referred to as visual effect functions. 

[0025] Associated with each selection function is a "visual effect" given by a set of visual effect 
parameters. Each visual effect parameter is the statistical expected value of the visual effect 
functions, given by the equation 

10 V = <£>= fjjf(r,g,b)p(r,g,b)drdgdb 

where p(r,g,b) is the joint probability distribution of the input image. 

[0026] FIG. 4 shows transform visual effect parameters. The definition of the visual effect 
parameters is given by an image probability distribution. Since the probability distribution of the 
output image can be calculated directly from the transform, we can associate the visual effect 
15 parameter with the transform for a given input. By definition, the output visual response 
parameter is 

V'= fffer',g',b')p'(r',g',b')dr'dg'db' 

We can calculate the output probability distribution from the transform and the input probability 
distribution as 

20 p , (r\g',b') = p(tinv(r',g , ,b')) 

[0027] Introducing a change of variables, we get 

V'= Jft(t(r,g,b))p(r,g,b)drdgdb 

This remarkable results allows us to associate the visual effect parameters with a transform and 
an input image. For the special case that the input image has uniform probability distribution, 
25 the uniform visual effect parameter for a transform is given by: 

Vt= JJfet(r,g,b)) drdgdb 
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[0028] FIG. 5 shows transform parameter set and visual effect parameter set conversion. This 
drawing shows the derivation of two estimator functions. The first estimates the visual 
parameters as a function of the synthesis parameters, and the second estimates the inverse 
function, the synthesis parameters as a function of the visual parameters. Associated with each 
5 estimate is an error measurement and the estimators are selected to minimize the error. 

[0029] FIG. 6 shows composing a sequence of parametric transforms into a single transform. 
This drawing shows the method of combining a sequence of parametric transforms into a single 
parametric transform. First, a v-array is constructed consisting of the set of p-arrays for each 
transform in the sequence. The reduction algorithm on page 13 of the Provisional Application, p 
10 = K(V) reduces the v-list to a p-list. This p-list is used to synthesize an equivalent transform. 

[0030] FIG. 7 shows scalar composition. Page 7 of the Provisional Application, section 
"Fundamental Definition and Algorithms," subsection "Analytic selective transforms" presents 
the notation and definition for a set of analytic selective functions. In most of the figures, the 
selective functions are equivalent to the visual effect functions. 

15 [0031] FIG. 8 shows simulation of a sequence of arbitrary transforms. This drawing shows a 
sequence of arbitrary transforms used to process an input image to an output image. It is desired 
to find a single transform that has the equivalent results as the sequence of transforms. This 
method calculates the visual effect parameters for the output image, estimates the corresponding 
p-array using the estimator in FIG. 5, and synthesizes a parametric transform. 

20 [0032] FIG. 9 shows the simulation of a sequence of arbitrary transforms. The method shown in 
FIG. 8 requires that an output image is obtained and used to calculate the visual effect parameter 
array. FIG. 9 shows a method where the output image is not required. In this figure, the visual 
effect parameters for each transform are measured, then used to estimate a corresponding p- 
array. The resulting set of p-arrays forms a v-array which uses the reduction algorithm, p = K(v) 

25 (page 13 of the Provisional Application) to derive the corresponding p-array which is used to 
synthesize and equivalent transform. 

[0033] FIG. 10 shows the synthesis of a sequence of visual intents. The method shown in FIG. 9 
requires that the output visual effect parameters from each of a sequence of transforms be 
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calculated or measured. In this figure, the user simply specifies the desired visual effect 
parameters for each stage. From this a transform is synthesized as in FIG. 9. 

[0034] The specification of the desired visual effect parameters is facilitated by using the 
"Unified User Interface" presented on page 14 of the Provisional Application. Note that that 
5 using a sequence of visual intents gives an intuitive and powerful method of specifying very 

complex transforms. This is a significant aspect to the authoring and editing capabilities set forth 
in the Provisional Application. 

[0035] FIG. 1 1 shows the execution of a selective image transform. Page 13 of the Provisional 
Application, section "Fundamental Algorithms for an Image Processing System" subsection 
10 "Execution" states the following equations for calculating a v-array. 

x' = t(K(G(S(x),v)), x) , where 

x = an input pixel value, typically x = (r,g,b) 
x' = an output pixel value, x' = (r',g',b') 
t is the parametric transform 
15 v is a v-list consisting of n p-list structures 

S(x) = (sj(x),. . .,Sn(x)) is a complete set of fundamental selectors 
G is the scalar combination algorithm 
K is the reduction algorithm that maps V into a p-list 
[0036] FIG. 12 shows the derivation of a v-array. Page 13 of the Provisional Application, 
20 section "Fundamental Algorithms for an Image Processing System," subsection "Combining 
processes at execution time" derives the equations for calculating a v-array. 

[0037] The algorithm for combining the q-array to a single v-array for execution is 

V = K(G(A,Q), where 

G is the scalar combination algorithm 
25 K is the reduction algorithm that maps Q into V 

The combined execution transform thus becomes: 
x»=t(K(G(S(x),K(G(A,Q))), x) 
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[0038] While the above is a complete description of specific embodiments of the invention, the 
above description should not be taken as limiting the scope of the invention as defined by the 
claims. 
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